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Abstract —  Asymmetric  threats  pose  a  difficult  challenge 
to  situational  awareness  systems.  Current  approaches  for 
predicting  or  even  detecting  an  asymmetric  threat  rely 
heavily  on  human  knowledge,  creating  scalability  issues 
due  to  the  vast  amount  of  data  to  be  analyzed.  Attempts 
to  automate  this  process  require  a  combination  of 
advanced  knowledge  representation  techniques  to  capture 
what  human  experts  know  about  the  domain  and 
inferential  reasoning  approaches  capable  to  work  with 
incomplete,  uncertain  data.  In  our  current  research,  we 
apply  a  verb-oriented  ontology  to  capture  actions, 
features,  indicators,  and  other  domain  elements  that  are 
relevant  to  asymmetric  threat  detection.  Then,  these 
elements  are  input  to  a  Bayesian  network  that  will 
calculate  the  posterior  probability  of  a  threat  given  the 
input.  As  in  any  complex  process,  evaluation  is  a  key  asset 
for  ensuring  that  nothing  is  neglected  and  partial  results 
are  consistent  with  the  expectations.  This  paper  describes 
our  approach  for  asymmetric  threat  detection  and 
emphasizes  how  we  are  leveraging  the  Uncertainty 
Representation  and  Reasoning  Evaluation  framework 
(URREF),  to  support  its  evaluation.  We  discuss  how  the 
sources  of  uncertainty  are  identified  and  how  we  assess  its 
impact  to  the  outcome  of  the  detection  system. 

Keywords:  asymmetric  threat,  Bayesian  network, 
uncertainty;  indicator,  ontology,  threat  analysis,  URREF 
criteria. 

I.  Introduction 

Until  a  few  years  ago,  the  vast  majority  of  researeh 
initiatives  within  the  field  of  data  fusion  were  foeused 
on  developing  solutions  for  symmetrie  military  warfare, 
in  whieh  equivalent  forees  were  expeeted  to  eomply 
with  international  eonventions  and  international  laws  on 
warfare.  As  a  result,  models  were  ereated  to  deseribe 
eomponents  of  the  enemy  forees  or  the  proeesses  used 
in  their  operations  and  field  manoeuvres,  see  e.g.  [15]. 
However,  the  emerging  eoneept  of  asymmetrie  warfare 
(see  e.g.  [19]  as  first  referenee)  beeame  a  major 
researeh  subjeet,  one  that  poses  serious  threats  to  both 
eivilian  and  military  faeilities. 

Asymmetrie  threat  refers  to  eireumstanees  in  whieh 
a  small  group  aims  to  destabilize  a  larger  and  more 
powerful  group,  while  avoiding  direet  eonfrontation  and 
using  irregular  forees.  The  asymmetrie  eonfliet  is 


eharaeterized  by  the  absenee  of  a  formal  eonfliet  area 
and  by  the  use  of  uneonventional  equipment:  prohibited 
weapons,  legitimate  weapons  employed  in  an  unlawful 
way,  improvised  deviees  or  even  eivilian  faeilities,  ef 
[17].  Aetions  eondueted  as  a  part  of  an  asymmetrie 
seenario  are  often  illegal  and  make  no  distinetion 
between  eivilian  (or  proteeted)  and  military  targets. 
Asymmetrie  adversaries  are  unpredietable  in  their 
behaviour,  and  deteetion  and  predietion  methods  for 
regular  warfare  are  usually  not  effeetive  against  them. 
New  proeessing  eapabilities  are  needed  to  support 
intelligenee  analysis  by  retrieving  patterns  of  hostile 
behaviour  or  elues  of  antagonistie  intentions  hidden  in  a 
large  amount  of  harmless  aetivity. 

This  paper  presents  a  user  oriented  approaeh  to 
deteet  and  foreeast  asymmetrie  threats.  The  approaeh  is 
based  on  Bayesian  Networks  (BN  [5])  and  has  been 
developed  for  and  is  integrated  to  the  AUGE  (German 
aeronym  for  “Automated  Threat  Deteetion”)  system,  a 
demonstrator  for  automatie  threat  reeognition  built  by 
lABG  in  a  projeet  sponsored  by  German  Air  Foree’s 
Transformation  Centre  [1],  [2].  The  projeef  s  goal  was 
to  provide  automated  support  for  J2  analysts  to  deteet 
asymmetrie  threats.  J2  analysts  are  responsible  for  the 
generation  and  assessment  of  the  situational  pieture  in 
large  military  units.  An  important  aspeet  of  the  projeet 
was  to  ensure  usability,  taking  into  aeeount  the  need  to 
seamless  eombine  the  standard  proeesses  within  the 
(GE)  J2  organization  with  the  experts’  methodologies 
used  to  perform  their  reasoning  and  assessment 
proeesses.  The  paper  is  struetured  as  follows:  The  next 
seetion  presents  approaehes  related  to  asymmetrie  threat 
analysis.  Seetion  III  introduees  our  model  for  threat 
analysis,  while  seetion  IV  presents  its  formalization  and 
the  implementation  of  the  user-eentred  approaeh. 
Uneertainties  related  to  this  approaeh  are  introdueed  in 
seetion  V,  and  their  assessment  based  on  the  URREF 
eriteria  is  diseussed  in  seetion  VI.  Conelusions  and 
direetions  for  future  work  end  this  paper 

II.  Related  work 

Asymmetrie  threat  is  an  emerging  eoneept,  related 
to  notions  sueh  as:  hostile  intent  [16],  hostile  aetivities 
[12],  suspieious  aetivities  [14]  or  even  anomalies  and 


the  so-called  “out  of  ordinary”  activities,  e.g.  [13]. 
Recently,  the  concept  has  been  employed  in  an  explicit 
manner  by  authors  such  as  Singh  et  al.  [14]  and 
Valenzuela  et  al.  [10]. 

One  of  the  first  solutions  proposed  to  address 
asymmetric  threats  is  the  AHEAD  (Analogical 
Hypothesis  Elaborator  for  Activity  Detection)  approach 
described  in  [12].  Its  authors  developed  a  domain- 
independent  method  for  hypothesis  elaboration,  taking 
structured  evidence  and  hypothesis  about  the  activities 
of  an  asymmetric  adversary  as  input.  The  method’s 
output  is  a  semantic  argument  supporting  or  rejecting 
the  hypothesis  that  combines  case-based  and  analogical 
reasoning  techniques.  This  solution  is  designed  to  assist 
the  user  in  testing  the  hypothesis  threat;  and  provides 
additional  information  in  the  form  of  arguments  to 
ascertain  the  validity  of  the  threat.  While  this  work 
focuses  on  hypotheses  elaboration,  various  solutions 
were  proposed  for  asymmetric  threat  detection  or 
prediction.  Among  them,  Singh  et  al.  consider  the 
identification  of  asymmetric  threats  in  relation  to 
anomaly  detection  [13].  An  anomaly  is  an  event  in 
which  the  distribution  of  observations  is  different  before 
and  after  an  unknown  onset  time.  Hidden  Markov 
models  are  used  to  model  patterns  of  asymmetric 
threats,  and  a  transaction-based  probabilistic  model 
allows  for  quick  identification.  Based  on  this  approach, 
the  AS  AM  (Adaptive  Safety  Analysis  and  Monitoring) 
system  was  developed  in  order  to  assist  analysts  to 
detect  asymmetric  threats  and  to  predict  possible 
evolutions  of  suspicious  activities.  The  system  is 
described  in  [14],  while  [17]  explains  its  use  to  model 
terrorist  events. 

Genshe  et  al.  propose  a  solution  for  asymmetric- 
threat  detection  and  prediction  based  on  advanced 
knowledge  base  and  stochastic  (Markov)  game  theory 
[11].  Asymmetric  threats  are  detected  and  grouped  by 
intelligent  agents  and  their  intentions  are  predicted 
using  a  decentralized  Markov  game  model.  The  method 
exploits  both  domain  knowledge  and  evidence  about  the 
current  situation,  while  their  solution  is  able  to  take  into 
consideration  the  adversary’s  decision  processes. 

Several  research  efforts  take  a  different  perspective 
and  aim  to  predict  asymmetric  threats  by  exploiting 
symbolic  sources,  such  as  intelligence  reports.  Chan  et 
al.  [9]  proposed  the  ATRAP  (Asymmetric  Threat 
Response  and  Analysis  Program),  a  set  of  tools  for 
annotating  and  automatically  extracting  entities  and 
relationships  from  documents.  Once  identified,  these 
elements  can  be  exploited  to  predict  adversaries’  future 
courses  of  action  by  creating  situational  threat  templates 
and  applying  customized  prediction  algorithms. 

Another  solution  based  on  templates  is  described  in 
[10].  Authors  developed  a  predictive  model  in  order  to 
automatically  survey  coded  hypotheses  (templates 
created  by  the  intelligence  community)  by  providing 
information  assessment  and  confidence  evaluation  from 


non-numerical  data.  The  predictive  model  is  composed 
of  different  parts:  information  retrieval,  assessment  of 
the  retrieved  information;  and  score  propagation.  The 
model  is  traceable,  transparent,  and  designed  for 
human-in-the-loop  data  fusion. 

In  our  approach,  asymmetric  threat  analysis  is 
considered  a  human-centred  task,  taking  advantage  of 
iterative  interventions  of  various  experts  to  create  the 
most  complete  model.  The  solution  is  designed  to 
support  both  hypothesis  elaboration  and  asymmetric 
threat  detection  and  prediction,  by  jointly  exploiting 
domain  knowledge  and  context  issues.  A  verb  ontology 
is  used  to  model  domain  knowledge,  making  the 
approach  easily  adaptable  to  various  application  fields. 

III.  A  MODEL  FOR  ASYMMETRIC  THREAT  ANALYSIS 

This  section  introduces  the  main  notions  used  for 
threat  analysis. 

A.  Components  of  threat 

The  threat  model  highlights  relevant  components  of 
threat  and  their  weighted  dependencies  according  to 
analysts’  opinions.  For  this  work,  it  is  important  to 
create  a  threat  model  that  closely  matches  the  mental 
model  used  by  analysts  when  analysing  possible  threats. 
The  model  corresponds  to  the  area  of  interest  of 
analysts  and  defines  a  threat  as  a  set  of  several 
components  (or  atoms).  Actors  within  the  Own  Area  of 
Interest  are  organizations,  groups  or  single  actors 
considered  as  possibly  threatening.  An  Actor-Type  (or 
several  Actor-Types)  is  assigned  to  each  actor  (e.g., 
terrorist  or/and  involved  in  organized  crimes).  An  actor 
evolves  within  its  Area  of  Influence  (i.e.  a  geographic 
area  or  a  cyber-area),  and  has  specific  Actor-Intentions 
(for  instance  “to  drive  away  ISAF  troops  from 
Afghanistan”  or  “to  get  rich  as  fast  as  possible”).  The 
intentions  can  be  effectuated  by  choosing  Option  for 
Action  (i.e.  to  perform  a  bomb  attack  at  a  market  place). 
The  Option  for  Action  can  be  realised  by  performing  a 
special  Action  Chain,  which  is  created  by  a  sensible 
sequence  of  (Single)  Actions.  To  perform  the  actions, 
the  Actor  must  use  available  material  and  personnel 
resources. 

According  to  the  model  above,  analysts  can 
elaborate  statements  describing  a  threat  as  follows:  “A 
specific  actor  A  pursues  an  Actor-Intention  I  and  has 
chosen  the  Option  of  Action  OA.  For  this  purpose,  his 
personnel  Resources  RP  are  performing  the  Action 
Chain  AC  with  the  Single  Actions  SA  using  the 
material  Resources  RM.  The  actor  has  the  Actor-Type  T 
and  acts  within  his  Area  of  Influence  AT” 

A  complete  description  of  threats  is  usually 
composed  of  several  personal  resources  using  various 
material  resources  to  perform  many  single  actions.  A 
proposition  of  the  analyst  can  restrict  one  or  some 
single  atoms  of  the  threat. 


Figure  1  Atoms  of  threat  and  their  dependencies 

The  atoms  of  the  proposition  are  elearly  not 
independent.  Dependeneies  identified  for  this  work,  are 
deseribed  in  Figure  1.  Moreover,  a  qualitative  weighting 
seheme  is  used  to  qualify  eaeh  dependeney  as  “very 
high”,  “high”,  “unknown”,  “low”  or  “very  low”.  For 
example,  the  aetor  “TALIBAN”  (organization)  has  a 
“very  high”  intention  to  drive  away  the  ISAF  troops 
from  Afghanistan,  but  has  a  “very  low”  intention  to  get 
the  Afghan  government  stabilized.  It  is  possible  that  the 
same  weight  value  will  be  assigned  to  several  intentions 
of  the  same  aetor. 


Tab.  1  deseribes  the  semanties  of  dependeneies: 


Dependency 

Semantics 

Actor  within  the  Own  Area  of 
Interest  ->  Type 

Type(s)  assigned  to  actors 

Actor  within  the  Own  Area  of 
Interest  ->  Area  of  Influence 

The  actor  has  freedom  of  action  in  the 
area(s)  of  influence. 

Actor  within  the  Own  Area  of 
Interest  ->  Intention 

Intentions  of  the  actor. 

Actor  within  the  Own  Area  of 
Interest  ->  Option  of  action 

Option  for  Action  elected  by  actor. 
Actors  often  prefer  special  OAs 
against  others. 

Actor  within  the  Own  Area  of 
Interest  ->  (personal  or 
material)  Resource 

Resources  (personal  or  material) 
available  for  actor 

Intention  ->  Option  of  action 

The  option  for  action  can  be  used  to 
effectuate  the  intention 

Option  of  action  ->  Action 
chain 

The  action  chain  can  be  used  to 
realize  the  option  of  action. 

Action  chain  ->  Single  action 

The  single  action  has  to  be  performed 
in  order  to  carry  out  the  action  chain 

Single  action  i  ->  Single 
action  k 

The  single  action  i  has  to  be 
performed  in  order  to  perform  the 
single  action  k. 

Resource  (personal)->  Single 
action 

Material  resource  ->  Single 
action 

The  personal  resource  is  able  to 
perform  the  single  action. 

The  material  allows  performing  the 
single  action. 

B.  Indicators  of  threat 

Indieators  are  defined  as  outeomes  of  intelligenee 
sourees  eonveying  evidenee  for  a  partieular  threat.  They 
eorrespond  to  conditions  of  suspicion,  or 
“signatures”  of  threatening  behaviour  and  several 
intelligenee  sourees  ean  be  eonsidered  (HUMINT, 
SIGINT,  IMINT  and  OSINT).  The  set  of  indieators 
offers  a  basis  for  searehing  evidenee  on  threats.  For 
instanee,  indieators  for  the  atomie  proposition  for  the 
aetion  “Opponent  reeonnoitres  the  own  eamp  by  eovert 
observation”  eould  be:  “Children  are  playing 
eontinuously  in  front  of  the  eamp”  OR  “A  sales  booth  is 
implemented  in  front  of  the  eamp”  OR  “A  person  is 
regularly  passing  the  eamp”  OR  ete. 

Users  ean  define  indieators  for  every  atomie 
proposition  of  the  threat.  Most  of  the  indieators  are 
ereated  for  single  aetions  or  resourees,  but  it  is  also 
possible  to  define  other  indieators  (e.g.  for  intention  to 
extraet  information  from  manifestos). 

C.  Hypothesis  of  threat 

Hypotheses  are  assumptions  that  explain  speeifie 
threats.  An  example  of  a  hypothesis  is  “I  guess  the 
TALIBAN  have  the  intention  to  unsettle  the  ISAF 
troops  by  ehoosing  the  eourse  of  aetions  lED  attaek 
together  with  subsequent  assaults”.  A  semi-automated 
approaeh,  deseribed  below,  was  developed  to  support 
experts  modelling  and  analysts  deteeting  threats. 

IV.  A  USER-CENTRED  APPROACH  TO  DETECT 
THREATS 

We  propose  a  general  arehiteeture  for  asymmetrie  threat 
deteetion  allowing  different  types  of  users  to  model 
various  threats,  to  identify  threat  indieators,  and  to 
elaborate  and  test  several  hypothesis  explaining  threats. 

A.  Using  Bayesian  network  (BN)  to  model  threats 

For  this  work,  BNs  are  used  to  model  threats 
eomposed  of  several  eorrelated  atoms.  This  formalism 
is  appropriate  as  it  allows  taking  into  aeeount  eausal 
dependeneies  of  threat  atoms  and  use  the  mutual 
exelusivity  of  some  parts  of  the  threat  model  in  order  to 
“deelare  away”  eompeting  propositions.  For  usability 
reasons,  it  is  neeessary  to  generate  the  Bayesian  model 
automatieally,  sinee  users  eannot  define  and  deal  with 
large  BNs  eomposed  of  several  hundreds  of  nodes. 
Weights  assigned  by  users  to  dependeneies  between 
atoms  of  threat  are  translated  to  eonditional  probability 
tables  (CPT),  and  their  semanties  is  preserved. 

The  strueture  of  the  BN  is  generated  from  the  threat 
model  by  eonsidering  the  faet  that  a  single  threat 
requires  multiple  resourees  and  ean  be  eomposed  of 
several  “single  aetions.”  Resourees  and  aetions  are 
translated  into  multiple  binary  non-exelusive  nodes.  The 
strueture  of  the  BN  assoeiated  to  a  threat  model  is 
sketehed  in  Fig.2. 


By  using  BNs  to  model  threats,  algorithms  required 
to  support  analyst’s  tasks  are  seleeted  from  the  BN 
algorithm’s  toolset. 


Fig.  2  Bayesian  Network  for  threat  detection 


B.  Formalization  and  identification  of  indicators  [2] 

A  verb-based  ontology  provides  a  formal  model  for 
indieators.  Verbs  are  eonsidered  important  for  this 
approaeh  as  threat-evidenee  is  related  to  aetivities  of 
opponents.  The  ontology  highlights  verbs  and  their 
assoeiated  frames  to  model  relevant  aetions  of 
opponents  and  their  assoeiated  eontext.  By  using  the 
ontology,  an  indieator  is  modelled  as  a  set  of  several 
verbs  together  with  their  eorresponding  frames  [7]. 

The  formal  model  of  an  indieator  eomprises  one  or 
several  verbs  from  the  ontology  together  with  their 
verb-frames,  whieh  ean  be  filled  with  additional 
qualifieations  to  sharpen  the  restrietions  to  be  matehed 
by  souree  information.  Indieators  are  identified  from 
textual  information  by  algorithms  using  linguistie 
methods  along  with  the  ontology  of  verbs.  Those 
algorithms  extraet  information  from  texts  and  translate 
it  into  verb  frames.  Therefore,  it  is  possible  to  eompare 
this  representation  to  indieators  defined  in  a  similar 
manner.  An  indieator  ean  be  matehed  exaetly  by  the 
information,  or  partially  when  the  strueture  of  the 
ontology  has  to  be  used  to  get  the  mateh  (e.g.  an 
indieator  seheme  eontains  a  red  Mereedes  and  the 
assoeiated  result  of  souree  information  contains  a  red 
car  at  the  same  part  of  the  scheme). 

C.  Elaboration  of  hypothesis 

Hypotheses  are  elaborated  for  both  model  and 
indicators.  Some  hypotheses  are  related  to  specific 
states  of  the  model,  and  in  this  case  it  becomes  possible 
to  trigger  changes  of  the  model  itself  When  related  to 
indicators,  a  hypothesis  allows  to  check  their  impact  on 
probability  values  of  the  BN  modelling  the  threat. 

D.  User-centred  identification  of  threats 

In  order  to  identify  asymmetric  threats,  various 
types  of  users  interact  with  the  model  in  order  to 
achieve  several  tasks,  as  described  hereafter. 


Subject  matter  experts  (SME)  define  and  maintain 
their  specific  part  of  the  threat  model.  Usually,  they 
have  knowledge  about  particular  factors,  e.g.  some 
SME  knows  “everything”  about  weapons  and  their 
distribution  in  Afghanistan.  The  design  of  the  threat 
model  allows  for  several  SME  to  improve  the  model  by 
adding  their  knowledge  regardless  of  one  another. 
However,  a  supervisor  is  in  charge  of  monitoring  the 
generation  of  the  threat  model,  and  it  can  perform 
causal  analysis  in  order  to  check  the  consistency  of  the 
model. 

The  source  specific  experts  are  responsible  for  the 
definition  of  the  indicators  and  the  generation  of 
indicator  matches  using  the  methodology  described 
above. 


Fig.  3  A  user-oriented  approach  to  model  and  detect  threats 


While  subject  matter  experts  create  the  threat  model, 
source  specific  experts  provide  indicators  to  assess 
evidence  of  one  or  more  elements  of  the  model. 
Therefore,  they  can  change  the  model  or  one  of  its 
states. 

Analysts  use  the  threat  model,  including  the 
indicators,  to  perform  causal  analysis  of  the  model, 
carry  out  a  diagnosis  and  build  and  assess  different 
hypothesis  explaining  the  threat. 

The  goal  of  causal  analysis  is  to  assist  in 
comprehending  different  factors  of  threats  along  with 
their  dependencies.  It  can  also  support  the  generation  of 
hypothesis  concerning  possible  threats.  A  diagnosis  is 
performed  in  order  to  identify  and  predict  threats, 
thanks  to  a  continuous  assessment  of  the  situation.  The 
diagnosis  is  calculated  using  the  evidence  generated  by 
the  indicator  matches.  Alerts  are  triggered  if  significant 
probability  values  are  assigned  to  some  atoms  of  the 
threat.  The  diagnosis  offers  a  means  for  a  long  term 
analysis  of  threats. 

Analysts  can  also  elaborate  hypotheses  about 
threats,  by  setting  a  priori  values  to  different  appropriate 
states  of  the  model,  or  hypotheses  about  evidences  by 


assigning  a  priori  values  for  indicators.  Hypotheses  can 
also  be  related  to  the  structure  of  the  model  and  can 
trigger  the  insertion  of  new  components  or  the 
elimination  of  existing  ones.  Analysts  can  also  assess 
the  hypotheses  by  taking  into  account  both 
contradictions  or  confirmations  of  hypotheses  with 
respect  to  evidence  and/or  domain  knowledge. 

V.  Uncertainty  Model  of  threat  identification 

The  process  of  threat  identification  using  the  user- 
oriented  approach  previously  described  is  affected  by 
different  types  of  uncertainties.  These  include  the 
quality  of  indicators  or  evidence  pieces,  the  way 
knowledge  is  handled  by  the  system,  and  the  form 
adopted  to  deliver  outcomes  to  users,  see  fig.  4. 


Fig.  4  Uncertainty  model  of  threat  detection 


A.  Uncertainty  of  inputs 

For  the  threat  analysis,  indicators  are  input  data,  and 
are  represented  as  binary  evidence  nodes  of  a  Bayesian 
network.  They  are  extracted  from  data  by  using  pattern¬ 
matching  approaches  that  usually  provide  uncertain 
results.  Therefore,  the  BN  is  enriched  thanks  to  so 
called  “soft  evidence”  [4],  in  the  form  of  a  probability 
value  p  assigned  to  each  indicator  node,  whose 
semantics  is  as  follows:  “The  proposition  represented 
by  the  indicator  is  true  with  probability  p.”  The 
probability  value  is  set  by  considering  both  the  perfect 
and  imperfect  matches  of  the  indicator  (see  [3]),  and  the 
quality  of  its  sources.  A  value  p  =  0.5  means  “no 
indicator  match”  due  to  the  binary  character  of  the 
evidence  node.  Therefore  it  is  also  possible  to  use 
information  contradicting  the  indicator  to  get 
probabilities  less  than  0.5. 

B.  Uncertainty  of  knowledge  handling:  Weighted 
Expectations  [3] 

Expert  knowledge  is  used  to  generate  CPTs  for  the 
BN,  by  adopting  the  “scale  based  distribution  retrieval.” 
This  is  a  two-step  approach:  first,  weight  values  defined 


by  users  are  transformed  into  scale  values,  which  are 
predefined  values  between  0  and  1.  The  translation  is 
carried  out  by  preserving  the  sequence  and  the  distance 
of  the  weights  according  to  the  meaning  of  qualitative 
values.  In  the  second  step,  the  resulting  table  column 
values  are  normalized.  For  dependencies  of  factors 
with  many  states,  the  normalization  step  leads  to  small 
numerical  values,  even  if  the  qualitative  value  of  the 
dependency  was  “very  high”.^ 

CPTs  reflect  the  dependencies  between  states  of  two 
nodes.  To  finalize  the  CPT,  the  probability  of  nodes 
having  more  than  one  parent  is  estimated  by 
multiplying  dependency  tables  of  the  considered  nodes. 
In  this  case,  the  domain  expert  indicates  the  type  of 
parent  node  (e.g.  the  material  resource,  the  personal 
resource  or  the  action  chain  as  parent  nodes  of  the 
single  actions)  having  more  or  less  influence  on  the 
dependent  node. 

For  this  approach,  special  cases  are  dependencies 
between  nodes  of  the  threat  model  and  nodes 
corresponding  to  indicators.  The  semantics  of  such 
dependencies  is:  “If  the  state  of  the  node  is  ...  then  an 
indicator  match  should  be  detected”.  At  the  BN  level, 
this  is  represented  by  an  oriented  connection,  going 
from  threat  node  to  indicator  node,  while  the  weight  of 
this  dependency  and  the  calculation  of  CPT  are  as 
already  described.  It  is  important  to  keep  in  mind  that 
the  CPT  represents  weighted  expectations,  and  no 
“real”  probabilities. 

C.  Reasoning  uncertainties 

For  this  approach,  various  states  of  the  BN  represent 
the  assessment  of  one  threat.  Moreover,  every  node  of 
the  BN  has  a  discrete  probability  distribution.  Some 
nodes  have  multiple  exclusive  states  e.g.  for  “actor 
within  area  of  interest”;  binary  nodes  correspond  to 
personal  and  material  resources  and  to  single  actions  as 
well.  States  of  nodes  represent  propositions  of  threat 
atoms,  and  propositions  about  a  threat  are  therefore 
created  by  combining  them. 

Thus,  reasoning  uncertainties  are  related  to  the 
capacity  of  the  system  to  handle  complex  BNs  and 
provide  accurate  results  within  a  reasonable  amount  of 
time. 

D.  Outcome  related  uncertainties:  weighted  threat 
factors 

Output  uncertainties  are  due  to  transformations 
required  to  create  the  outcome  and  to  provide  this 
outcome  in  a  user-friendly  form.  After  creating  the 
weighted  BN,  diagnostic  algorithms  are  used  to 
compute  probability  values  of  node  states.  The  relative 
probability  value  of  an  atom  is  given  by  the  calculated 


^  This  should  not  annoy  the  BN  expert,  but  it  might  puzzle  domain 
experts. 


probability  distribution.  The  relative  probability  of  a 
threat  is  eomputed  by  eombining  the  probabilities  of  its 
atoms  respeetively.  Thus,  the  most  probable  and  also 
most  improbable  threats  ean  be  easily  extraeted.  The 
results  of  the  ealeulations  are  eonsidered  as  weighted 
threat  faetors  and  have  to  be  re-translated  into  the 
qualitative  values  of  the  user  weights,  in  order  to 
provide  a  user-friendly  form  of  results  as  statements 
sueh  as:  “It  is  very  probable  that  the  aetor  x  is 
responsible  for  the  eonsidered  threat  and  its  intension  is 
to 

VI.  ASSESSMENT  OF  UNCERTAINTY  USING  URREF 

In  this  ehapter  the  deseribed  approaeh  is  analysed 
based  on  measures  defined  by  the  uneertainty 
representation  and  reasoning  evaluation  framework 
(URREF),  whieh  is  depieted  in  Figure  5.  In  URREF^, 
eriteria  quantify  eaeh  type  of  uneertainty  previously 
identified  and  direetions  to  evaluate  them  are  proposed. 
Beeause  the  approaeh  has  a  strong  foeus  on  usability 
and  traeeability  of  results,  the  related  eriteria  are  of 
speeial  importanee. 

The  deseribed  approaeh  is  designed  so  that  the  data 
used  for  the  fusion  is  defined  by  the  users.  Therefore, 
the  analysis  of  the  approaeh  is  restrieted  to  the 
diseussion  whether  it  ean  eover  and  handle  the  relevant 
uneertainties.  If  the  approaeh  is  applied  in  a  real 
environment  it  is  expeeted  that  the  eonfiguration  data  of 
the  model  will  eontinuously  be  improved  by  the  SME 
and  souree  experts  by  diseussing  the  results  of  the 
system  in  interaetion  with  the  analyst. 

It  is  therefore  neeessary  to  use  a  data  set  eovering  a 
relatively  long  seenario  period  to  evaluate  the  system 
with  experimental  seenario  and  data.  As  a  eonsequenee, 
a  large  data  set  is  required.  Nevertheless,  the  presented 
analysis  of  different  uneertainty  levels  assoeiated  to 
various  elements  of  this  model  allows  us  to  have  a  first 
assessment  of  the  overall  proeess. 

A.  Criteria  for  input  related  uncertainties 

Those  eriteria  are  intended  to  qualify  indieators 
provided  by  analysts  when  analysing  threats.  Eaeh 
indieator  is  assessed  inerementally,  using  values  of 
eredibility  and  relevanee  to  the  problem. 

Credibility  is  a  value  of  souree  information,  whieh  is 
the  basis  of  the  ealeulation  of  indieator  matehes.  The 
eredibility  of  information  is  usually  provided  by  its 
origin.  If  an  indieator  mateh  is  ealeulated  based  on 
souree  information,  its  eredibility  is  passed  on  to  the 
indieator  mateh.  An  evaluation  seheme  ean  be  used  to 
enable  different  analysts  to  evaluate  indieators  in  a 
similar  manner.  For  instanee,  NATO  standard  [8]  ean 
be  used  to  assess  indieators  provided  by  HUMINT  data. 


^  The  URREF  ontology  is  available  at  http://eturwg.c4i.gmu.edu 


Indieators  have  different  degrees  of  relevanee  to  the 
problem.  Firstly,  they  are  ereated  by  a  eomplete  or 
partial  mateh  of  the  information  and  the  pattern  defined. 
Seeondly,  the  CPT  linking  the  atom  of  the  threat  model 
to  the  indieator  also  represents  the  SME  assessment  of 
the  indieator  mateh ’s  relevanee.  Henee,  if  an  indieator 
mateh  arises;  it  is  also  possible  to  estimate  its  relevanee 
to  the  problem. 

The  approaeh  provides  an  internal  funetionality  used 
to  improve  the  aeeuraey  of  input  uneertainties.  If  there 
is  an  indieator  mateh  the  system  ean  present  the  original 
information  for  a  seleeted  indieator  mateh  together  with 
the  information  extraeted  using  the  linguistie  approaeh. 
It  ean  be  used  by  experts  to  assess  whether  the 
deseribed  values  are  adequate  whieh  enables  the  experts 
to  adapt  them. 

B.  Criteria  for  uncertainties  concerning  knowledge 
handling 

The  eriteria  for  knowledge-handling  are  relevant  to 
assess  the  way  SMEs  ereate  the  threat  model  thanks  to 
their  domain  knowledge.  This  was  one  of  the  most 
important  requirements  of  the  approaeh.  We  diseuss 
adaptability  and  simplieity  beeause  they  are  the  most 
relevant  eriteria  for  the  usability  of  the  knowledge 
handling. 

“Adaptability  eriteria  eneompass  the  ability  of  the 
representational  model  to  allow  for  different 
eonfigurations  of  it.  As  an  example,  an  adaptable 
representational  framework  would  have  most  of  its 
elements  eonfigurable  by  Subjeet  matter  Experts 
(SME)[20]”. 

The  approaeh  ean  be  eontinuously  adopted  during 
its  use  in  a  speeial  mission  taking  into  aeeount  the 
evolving  knowledge  of  the  SME.  Additionally  the 


model  should  also  be  easily  adaptable  to  different  types 
of  missions,  e.g.  Congo  and  Afghanistan.  Finally  the 
deseribed  approaeh  is  adaptable  not  only  to  asymmetrie 
threat  applieations  but  to  all  applieations  having  the 
same  strueture  of  the  threat  model,  e.g..  to  organized 
erime  and  homeland  seeurity  applieations  as  well. 

Simplieity:  “This  refers  to  the  ability  to  use  the 
uneertainty  management  eapability,  e.g.,  to  exeeute 
eommon  operations  (eonfigure  the  system,  enter 
evidenee,  proeeed  with  analysis,  ete.)  without  requiring 
deep  knowledge  about  the  inner  details  (mathematieal 
underpinnings  of  the  inferential  proeess,  algorithmie 
details,  ete.y’  (Rewording  of  definition  by  Kathryn 
Laskey)  [20]  This  was  one  of  the  approaeh’ s  key 
features  as  deseribed  in  ehapter  IV. 

C.  Criteria  for  reasoning  related  uncertainties 

Reasoning  eriteria  are  intended  to  eapture  both  the 
quality  of  the  model  ereated  by  different  experts  and  to 
provide  a  means  to  evaluate  the  inferenee  proeess 
performed  for  threat  deteetion. 

First,  the  model  ean  be  eharaeterized  by  its 
dissonanee,  whieh  oeeurs  as  model  is  ereated  and  used 
by  different  aetors.  For  instanee,  dissonanee  beeomes 
relevant  when  different  sourees  provide  eontradietory 
information  or  when  different  experts  ehange  the  model 
in  a  way  that  different  parts  of  the  model  beeome 
ineonsistent.  As  BN  are  used  to  model  threats,  the  level 
of  dissonanee  ean  be  estimated  by  using  BN  tools. 

Inferenees  used  to  deteet  threats  are  assessed  by 
their  sealability,  eomputational  eost  and  timeliness. 
Sealability  is  intended  to  eapture  the  size  and  the 
eomplexity  of  the  BN.  For  this  approaeh,  the  strueture 
and  the  number  of  the  threat  model’s  faetors  are 
eonstant,  but  nodes  have  an  important  number  of  states 
and  are  eonneeted  by  numerous  dependeneies. 
Therefore,  values  of  sealability  ean  be  estimates  by 
taking  into  aeeount  the  number  of  BN  nodes  and  the 
density  of  its  links. 

The  eomputational  eost  is  related  to  BN  intrinsie 
performanees,  and  it  ean  be  important  for  BN  having 
important  sizes. 

Timeliness  is  an  important  aspeet  to  be  eonsidered 
when  the  system  is  used  interaetively  for  hypothesis 
generation  and  assessment.  This  measure  is  related  to 
BN  intrinsie  performanees  and  to  the  eomplexity  of  the 
task  performed. 

D.  Criteria  for  output  related  uncertainties 

Results  of  this  approaeh  are  in  the  form  of  assertions 
deseribing  both  the  threat  and  its  assoeiated  probability. 

Therefore,  preeision,  interpretation  and  traeeability 
are  eriteria  deseribing  the  outeome. 


A  high  level  of  preeision  allows  users  easily 
identifying  the  most  probable  threats.  Even  if  the 
deseribed  translation  of  the  BN’s  probability  values  to 
qualitative  attributes  of  the  user  model  deereases  the 
preeision  of  results,  this  translation  is  neeessary  as 
intuitively  users  eneounter  diffieulties  while  analysing 
not  elearly  differentiated  values  of  probabilities. 

Interpretation  is  a  key  faetor  for  a  user-oriented 
approaeh.  The  approaeh  is  designed  to  provide  results  in 
a  user-friendly  form  by  translating  the  ealeulation 
results  into  the  user  model.  Therefore  the  outeomes  ean 
easily  be  interpreted  and  eompared. 

Traeeability  of  results  is  an  important  aspeet  for 
asymmetrie  threat  applieations,  as  the  user  has  the  entire 
responsibility  of  the  outeome  provided  and  therefore 
needs  to  eheek  the  final  result  manually  before  making 
further  use  of  it.  Traeeability  is  firstly  obtained  by  using 
BN.  Additionally  the  approaeh  is  designed  in  a  way  that 
the  user  is  able  to  eheek  if  the  result  is  sensible  by 
presenting  the  original  information,  the  indieator 
matehes  and  the  eausal  relations,  whieh  are  the  reasons 
for  the  final  result. 

VII.  Conclusion  AND  FUTURE  WORK 

This  paper  presents  a  user-oriented  approaeh 
developed  for  asymmetrie  threat(s)  deteetion.  User 
integration  is  eonsidered  a  key  feature  to  perform 
asymmetrie  threat  analysis  in  order  to  provide  an 
approaeh  through  whieh  the  user  will  trust  in  the  fusion 
results.  The  approaeh  aims  to  deteet  threatening  insights 
out  of  enormous  amounts  of  noisy,  seattered  and  partial 
data.  Our  solution  is  based  on  Bayesian  Networks,  and 
exploits  domain  knowledge  in  order  to  extraet 
indieators.  We  also  diseuss  the  uneertainty  model 
related  to  our  approaeh  and  diseuss  the  assessment  of 
uneertainty  by  using  URREF  eriteria. 

Performing  human-in-the-loop  evaluation  and 
validation  is  a  direetion  for  future  work.  A  set  of  real 
data  or  a  seenario  with  realistie  data  sets  will  be  used, 
and  results  of  the  approaeh  will  be  eompared  to 
evolving  reality.  The  main  assessment  eriterion  will  be: 
“Does  the  approaeh  improve  the  eapability  of  users  to 
deteet  threats.”  For  this  validation  proeess,  URREF 
eriteria  ean  be  used  in  order  to  reeeive  a  more  detailed 
analysis  of  results.  This  detailed  analysis  ean  be  applied 
to  the  different  parts  of  the  approaeh  and  to  the 
intermediate  results  of  the  threat  analysis. 

Future  work  eould  also  eonsist  of  improvement  of 
the  approaehes’  results,  thanks  to  the  assessment  of 
uneertainty  eriteria.  In  this  ease,  uneertainty  eriteria  will 
be  used  to  evaluate  the  relevanee  of  different  elements 
of  the  model  in  order  to  ignore  non-relevant  features, 
whieh  eould  improve  the  aeeuraey  of  the  solution. 

An  additional  promising  applieation  of  the  URREF 
eriteria  is  the  development  of  a  quality  assuranee 


functionality  to  be  used  by  the  persons  whieh  are 
responsible  to  assure  the  quality  of  the  threat  model.  As 
already  deseribed  the  threat  model  is  adapted  by 
different  person  with  different  responsibilities  during  its 
usage.  There  is  an  inherent  danger,  that  the  model 
beeomes  worse  due  to  ehanges  e.g.  by  adding 
ineonsistent  and  /  or  redundant  domain  knowledge. 
URREF  eriteria  ean  be  ealeulated  to  support  quality 
managers  to  deteet  degradations  of  the  model  quality. 

A  different  direetion  for  future  work  eoneems  the 
improvement  of  teehniques  used  to  extraet  indieators. 
For  instanee,  user  interventions  required  to  validate  the 
extraeted  information  eould  be  used  to  provide 
additional  information  about  their  quality,  whieh  ean  be 
taken  into  aeeount  by  the  system  automatieally. 
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